In common video-coding methods such as MPEG or the like, differences between two successive pictures are generally determined in order to compress the amount of video data information. In such an inter-picture prediction method (called the "inter-picture coding method" hereinbelow), a present picture is predicted according to a previous picture, and a predicted difference is transmitted.
However, when an error is incorporated into a picture passing through a transmission path, thus is rendering the relevant data erroneous, the corresponding picture at the receiving side is also erroneous because the picture is constructed using erroneous data. As a further problem, the next picture is predicted based on the error-containing picture, and the error is propagated to subsequent pictures.
Such a situation in which an error is incorporated into video data occurs when a reading error from a storage medium such as a CD-ROM exists, or a transmission error via a transmission path exists. In particular, significant transmission errors occur if a radio transmission path is used.
As the first conventional example of the video transmission method using such an error-prone transmission path, an intra-picture coding method will be explained, in which if an error is detected at the receiving side, the receiving side informs the video sending side of the situation, and the sending side which received the information encodes video data without using the inter-picture coding method.
FIG. 18 shows an example of a combination system arrangement of the above first conventional example and a general video coding method in which motion compensation and discrete cosine transformation (DCT) are combined.
First, a difference between an input picture and a reference picture is calculated by subtracter 101, and the DCT is performed at DCT section 102. The DCT coefficient as a result of the DCT is quantized by quantizer ("Q") 103 and is temporarily stored in buffer ("buff.") 109. The stored data are read out through control by error controller ("error cont.") 110 and are transmitted through transmission path 120. The signal quantized by quantizer 103 is inverse-quantized by inverse quantizer "Q.sup.-1 " 104 and is then subjected to inverse DCT in inverse DCT section ("IDCT") 105. To the output from section 105, the same reference picture signal as the signal used at the subtracter 101 is added and the added result is stored into picture memory ("PM") 107. Picture-differential controller 108 reads out a reconstructed picture of one picture before and sends it as an reference picture signal to subtracter 101. When the error controller 110 is informed of error detection from the receiving side, the controller 110 interrupts the inputting of a reference picture signal via picture-differential controller 108 into subtracter 101 and switches the coding method to the intra-picture coding.
Whether or not a receiving error exists in the information transmitted through transmission path 120 is determined by error detector 111 at the video receiving side. If no error is detected, the information is inverse-quantized by inverse quantizer ("Q.sup.-1 ") 112 and then inverse DCT is performed by inverse DCT section ("IDCT") 113. To the output from section 113, a received picture of one picture before, stored in picture memory ("PM") 115, is added as an reference picture signal by adder 114. The added result is output to a monitor or the like, and is simultaneously stored in PM 115. If a receiving error was detected by the error detector 111, this fact is communicated to the video sending side.
As the second conventional example of the video transmission method using an error-prone transmission path, another method will be explained in which error propagation is prevented by changing a reference picture used for calculating an inter-picture difference at the time of encoding. That is, in this method, if an error is detected in a received picture at the decoding side, the fact that the error was detected, and the relevant picture number (or temporal reference, etc.) or the newest picture number which was correctly received and decoded are communicated to the encoding side, and the reference picture used when the next picture is encoded is changed from the error-detected picture to the picture which was correctly received last.
FIG. 19 shows an example of a combination system arrangement of the above second conventional example and a general video coding method in which motion compensation and the DCT are combined. Here, elements 201-215 and 220 in FIG. 19 respectively correspond to elements 101-115 and 120 in FIG. 18.
Distinctive features of this second example in comparison with the first conventional example are that plural PMs are provided, as shown by reference numerals 207 and 215, and that picture-memory (PM) storing sections 216 and 218 and picture-memory (PM) selectors 217 and 219 are provided.
First, construction and operation of the video sending side will be explained.
First, a difference between an input picture and a reference picture is calculated by subtracter 201, and the DCT is performed at DCT section 202. The DCT coefficient as a result of the DCT is quantized by quantizer 203 and is temporarily stored in buffer 209. The stored data are read out through control by error controller 210 and are transmitted through transmission path 220. On transmission, the error controller 210 also sends the number of the encoded picture and the number of a picture which was used as the reference picture.
The signal quantized by quantizer 203 is inverse-quantized by inverse quantizer 204 and is then subjected to inverse DCT in inverse DCT section 205. To the output from section 205, the same reference picture signal as the signal used at the subtracter 201 is added and the added result is sent to PM storing section 216.
The PM storing section 216 stores the transmitted data (pictures) into PMs 207 in the order PM1.fwdarw.PM2.fwdarw.. . . .fwdarw.PMn.fwdarw.PM1.fwdarw.. . . , and informs PM selector 217 of the number of the picture and information on the PM to which the relevant picture is stored. The PM selector 217 retains the information of the correspondence relationship between each PM and the relevant picture number, which was communicated from the PM storing section 216, and selects one of the PMs (PM1-PMn) 207 according to a signal from error controller 210. The PM selector 217 then reads out the picture stored in the selected PM and sends the contents of the PM as the reference picture signal to subtracter 201.
When error controller 210 is informed of the error detection and the error-containing picture number via transmission path 220 from error detector 211 of the video receiving side, the controller 210 communicates the number of the error-containing picture detected at the video receiving side to PM selector 217. The PM selector 217 reads out a picture of the newest number prior to the communicated number from the relevant PM 207. The selector sends the read-out picture to picture-differential controller 208 and communicates the number of the picture to error controller 210 of the video sending side. The error controller 210 informs the video receiving side of the communicated picture number as the picture number of the reference picture, with the number of the picture being encoded.
If no picture prior to the picture of the number communicated from the error controller 210 is stored in PMs 207, PM selector 217 interrupts the inputting of the reference picture signal via picture-differential controller 208 into subtracter 201, and switches the coding method to intra-picture coding. In addition, error controller 210 ignores the signal indicating the number of an error-containing picture, repeatedly sent from error detector 211 of the receiving side (that is, the signal repeatedly sent until the picture which was encoded using the changed reference picture is received at the video receiving side).
Hereinafter, construction and operations of the video receiving side will be explained.
Whether or not a receiving error exists in the information (or data) transmitted through transmission path 220 is determined by error detector 211. If no error is detected, the information is inverse-quantized by inverse quantizer 212 and then inverse DCT is performed by inverse DCT section 213. On the other hand, error detector 211 informs PM selector 219 of the picture number which was used as the reference picture signal for the present encoded picture and which is communicated from error controller 210. The error detector simultaneously reads out the picture of the communicated picture number from PMs 215 and sends the picture to adder 214. The adder adds the sent reference picture signal to the received signal and outputs a result to a monitor or the like, and also sends the result to PM storing section 218.
The PM storing section 218 performs the same operations as PM storing section 216 of the video sending side, that is, stores the received data (or pictures) into PMs 215 in the order PM1.fwdarw.PM2.fwdarw.. . . .fwdarw.PMn.fwdarw.PM1.fwdarw.. . . , and informs PM selector 219 of the number of the picture and information on the PM to which the relevant picture is stored. The PM selector 219 retains the information of the correspondence relationship between each PM and the relevant picture number, communicated from the PM storing section 218, and selects one of the PM1-PMn according to a signal from error detector 211. The error detector 211 then reads out the picture stored in the selected PM and sends it to adder 214.
When a receiving error is detected, the error detector 211 informs the video sending side of the fact of error detection and the error-containing-picture number or the number of the newest picture which was correctly received and decoded. In addition, the error detector 211 stores and retains the error-containing-picture number and repeatedly informs PM selector 219 of the retained picture number until the detector receives video data which was encoded using data belonging to a picture number prior to the stored picture number as the reference picture, or until it receives intra-picture encoded video data.
FIG. 20 is a diagram for explaining an operational example of the second conventional example, a specific time-series operational example using respective "4" PMs 207 and 215.
In this figure, reference numeral 301 indicates the number of each picture being encoded at the video sending side, and reference numeral 303 shows the picture numbers indicating each content of PMs (PM1-PM4) 207 at the starting time of the encoding of the next picture after the encoding of a picture is completed. Reference numeral 302 indicates the number of each picture under decoding at the video receiving side, and reference numeral 304 similarly shows the picture numbers indicating each content of PMs 215 at the starting time of the decoding of the next picture after the decoding of a picture is completed. In this time-series arrangement, processed picture number 302 at the decoding side is shifted from processed picture number 301 at the encoding side because of a necessary transmission time. Regarding numbers appended to (solid) arrows which indicate video data, "9/8" (as an example) indicates (video) data obtained by encoding the picture of picture number "9" using the picture of picture number "8" as the reference picture. Actually, the amount of video data is very large and thus continuously flows from the sending to the receiving side. However, for the purpose of making the figure clear, only the last part of the video data is indicated by each (solid) arrow.
The present operational example of FIG. 20 shows a case in which data of picture number "10" contain an error. When error detector 211 detects an error, the detector sends a NACK signal (refer to reference numeral 305 in FIG. 20) including the error-containing-picture number and stores the number "10", the error-containing-picture number. This NACK signal (10N) is received by error controller 210 while the video sending side processes the picture of number "12". The error controller 210 controls PM selector 217 in a manner such that the selector selects the number "9", the newest number prior to the error-detected picture number "10", and that the picture of the next number "13" is encoded using the picture of the selected number as the reference picture. Simultaneously, the error controller informs the video receiving side that picture "9" was used as the reference picture (refer to reference numeral 306 in FIG. 20). When error detector 211 receives the signal of this information, the detector directs PM selector 219 to select and read out the picture of number "9" as the reference picture. In addition, error detector 211 repeatedly sends a NACK signal including picture number "10" (10N) with respect to video data of picture numbers 11 and 12, even though those video data themselves include no error because the picture as the reference picture (for them) has a number larger than "10".
In the present operational example, the error is propagated from error-containing picture "10" to picture "11" which uses picture "10" as the reference picture and to picture "12" which uses picture "11" as the reference picture, but is not propagated to picture "13" which uses the changed reference picture. In this way, this second conventional example has an advantage in that even if an error occurs, propagation of the error can be prevented without switching the coding method to the intra-picture coding.
In the above first conventional example, by performing the intra-picture coding, the amount of data necessary for transmitting a picture is remarkably increased in comparison with a no-error case, which causes a problem in that the number of transmitted pictures is decreased and the quality of decoded pictures is degraded because the quantization step size is increased. In addition, because of the large amount of data, there is a high probability that intra-picture encoded data will again include an error; thus, a vicious circle occurs in which intra-picture encoding must be repeatedly performed.
In the above second conventional example, if the arrival of the NACK signal sent from the video receiving side to the video sending side is delayed due to any problem, it takes substantial time for the video sending side to change the reference picture, whereby there occurs a problem in that an appropriate reference picture no longer exists in the PMs when the change of reference picture is performed. Such a problem occurs when (i) an error enters into the NACK signal during its transmission and the video sending side cannot recognize the signal, (ii) a time necessary for transmission is long, (iii) the transmission time includes fluctuation, and (iv) the amount of each encoded data is not fixed. In such situations, only the intra-picture coding method can block error propagation, even in the second conventional example; thus, problems similar to the first conventional example also occur in the second example.
FIG. 21 is a diagram for showing an operational example having the same conditions as the example shown in FIG. 20, in which an error was generated in a signal sent from the video receiving side to the video sending side, and thus the sending side could not recognize the sent signal.
In this operational example, reference numeral 401 indicates the number of each picture being encoded at the video sending side, and reference numeral 403 shows the picture numbers indicating each content of PMs 207 at the starting time of the encoding of the next picture after the encoding of a picture is completed. Reference numeral 402 indicates the number of each picture under decoding at the video receiving side, and reference numeral 404 similarly shows the picture numbers indicating each content of PMs 215 at the starting time of the decoding of the next picture after the decoding of a picture is completed. In this time-series arrangement, processed picture number 402 at the decoding side is shifted from processed picture number 401 at the encoding side because of a necessary transmission time.
The present operational example shows a case in which data of picture number "10" contain an error, as in the example shown in FIG. 20. When error detector 211 detects an error, the detector sends a NACK signal including the error-containing-picture number (refer to reference numeral 405 in FIG. 21). Here, an error occurred in this NACK signal and error controller 210 could not recognize the erroneous signal; thus, the video sending side encodes the picture of picture number "13" using the picture of number "12" as the reference picture. The video receiving side next receives data of picture number "11" which was encoded using the error-containing picture of number "10" as the reference picture; thus, a NACK signal including picture number "10" is sent (refer to reference numeral 406 in FIG. 21). When this NACK signal (10N) is received by error controller 210 while the sending side processes the picture of number "13", the error controller 210 controls PM selector 217 to select a number which is the newest prior to the error-detected picture number "10" so as to encode the picture of number "14". However, at this point, no picture prior to number "10" exists in PMs 207. Therefore, the picture of number "14" must be encoded using the intra-picture coding method (refer to reference numeral 407 in FIG. 21).
Next, FIG. 22 is a diagram for showing an operational example having the same conditions as the example shown in FIG. 20, in which the necessary transmission time is long.
As in the case as shown in FIG. 20, an error is generated in video data of picture number "10" and the NACK signal (10N) communicating the situation is received by error controller 210. Here, the arrival of this NACK signal is delayed (refer to reference numeral 505 in FIG. 22); thus, the picture next decoded at the video sending side is transferred to picture number "15" and at the time of encoding of the picture "15", no picture prior to picture number "10" exists in PMs 207. Therefore, the picture of number "15" must be encoded using the intra-picture coding method.